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(54) Generation of test suites for interoperability of reactive communication systems 



(57) A method of generating test sequences for de- 
termining interoperability of communication systems 
when connected to one another, includes defining gate- 
ways between the connected systems and correspond- 
ing end users of the systems. Certain transitions involv- 
ing operation of more than one gateway associated with 
a calling user and a called user, are determined. The 
connected systems are tested for operation at those 



transitions. In an illustrated embodiment, a state ma- 
chine corresponding to the connected systems is de- 
fined, the machine having a number of defined states, 
and transitions between the states. Those transitions in- 
volving operation of more than one system gateway as- 
sociated with a calling user and a called user, are deter- 
mined. The connected systems are tested for operation 
at at least some of those transitions. 
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Description 

Field Of The invention 

[0001] This Invention relates to a method of generating a set of test sequences to confirm proper operation of inter- 
connected communication systems. 

Discussion Of The Known Art 

[0002] Generation of test sequences to determine whether or not a given communication system confomis to a 

specification or design requirement, is common practice. Automated test equipment available on the market is usually 
rented or leased by a system operator to test a system for proper operation when necessary. Such equipment is typically 
configured to perform a number of key tests in accordance with a program written specifically for the system. 
[0003] It is believed that Internet protocol (IP) will play a large, if not dominant, role in future public telephony networks. 
Such networks are expected to handle not only voice, but video and other information and data as well. It is therefore 
important that existing data-centric and public switched telephone networks (PSTNs) be capable of operation when 
connected with one another through so-called "gateways". The ability of such networks or systems to operate properly 
when inter-connected is typically referred to as system "interoperability". 

[0004] With respect to providing voice calls over an IP network ("Voice over IP" or "VoIP"), the first VoIP systems 
provided voice communications over a local area network (LAN) for end-users seated at personal computers. This 
proved unsatisfactory for several reasons. For example, voice quality was poor due to a "best-effort" standard for data 
packet delivery in the LAN environment. Also, there was no way to direct a voice call to a user connected to a PSTN, 
thus limiting the number of possible calls. And, a called end-user PC might not be connected to the network when the 
call was placed. 

[0005] Because some providers believed IP telephony might circumvent long distance user rates due to the absence 
of tariffs on local Internet access, local user access gateways to PSTNs were created in each locality to be served, 
with a private Internet providing a long-distance backbone for voice calls. LEG'S became interested in a similar strategy, 
but instead of using a gateway to connect local lines to an IP network, gateways were used to connect their central 
offices to an IP network, thus replacing a circuit-switched network backbone with an IP network. Finally corporations 
maintaining large data networks became interested in combining their voice and data networks by running Voice over IP. 
[0006] In view of the above, the existing diversity of Vol P implementations results in diverse communications systems 
that may not be able to interoperate when connected together. In order for these existing communication systems to 
remain effective, they must be able to intenwork with the PSTN and with other VoIP products. 

The Standards 

[0007] Three important Intemet telephone standards now exist. Initially, Standard H.323 for Internet Telephony re- 
flected the LAN bias of earlier Internet telephony systems. Subsequent standards address use of packet voice over 
wide-area networks and interwork with the PSTN. 

[0008] Standard H.323 version 2 is a current dominant standard for commercial products. A second standard, Ses- 
sion Initiation Protocol (SIP), is a proposed standard of the Internet Engineering Task Force. A third standard, Media 
Gateway Control Protocol (MGCP), has also been proposed to provide intenworking between Intemet telephone and 

the PSTN, 

[0009] The H.323 Standard is both an architecture and a standard for Voice over IP. It is an umbrella standard, 
specifying a collection of standards to be used by the components of the architecture which divides required functionality 
among gatekeepers, gateways, multipoint control units (MCUs), and endpoints. 

[0010] Specifically, endpoints are the initial creators and ultimate recipients of information streams. They need not 
comply with any part of the H.323 protocol, but if they do not, they must communicate with the network through a 
gateway that does comply with the protocol. An endpoint that complies with H.323 is called an H.323 terminal. 
[0011] An H.323 gateway provides communication between an H.323 network and a non-H.323 device or network. 
It may also provide communication with a second H.323 network. 

[0012] An H.323 gatekeeper handles address translations, for example, between telephone numbers and IP ad- 
dresses, andcontrols access to the network. A gatekeeper controls admission to a zone. Gateways, MCU's. and H. 
323 terminals must register with a gatekeeper, if one is present in their zone, and obtains permission to join a call. 
Optionally call signaling may also go through the gatekeeper There is no gatekeeper-to-gatekeeper standard, so that 
communication between multiple zones must be managed in an ad hoc fashion. 

[0013] A multipoint control unit provides conferencing capabilities in an H.323 network. It includes a multipoint con- 
troller, and (optionally), one or more multipoint processors. The multipoint controller handles signaling that detemiines 
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Who participates in a conference, and what information streams they send and receive. The multipoint processors 
provide centralized processing of information streams (audio, video, or data) from various parties to a conference. The 
processing can be mixing, switching, or anything supporting a specific conference connection. 
[001 4] An H.323 terminal or gateway uses a Registration, Admission, and Status (RAS) channel defined in the stand- 
ard to register with a gatekeeper, to locate other gateways and terminals, and to request pemriission to start or join a 
call. It may also route the call signaling through the gatekeeper. The call signaling protocol Is a Q.931 subset, also 
defined In Standard H.225.0. 

[0015] The actual media connections in a call and their types, are negotiated directly between gateways and/or 
endpolnts using Standard H.245. Once the endpoints agree, transport connections are also set up using the H.245 
standard. Transport connections can use any of a number of standards. For audio, Standard G.711 for uncompressed 
voice is required, but not always implemented. Other, optional audio Standards are G.723.1 and G.729. For video. 
Standard H.261 is required; and Standard H.263 is optional. For data, Standard T.I 20 Is required. 
[0016] Because of the many choices available when building an H.323-compliant system, and since requirements 
for communication between zones are not specified, It Is quite likely that H.323-compliant systems manufactured by 
different vendors will not interoperate. Further, the number of configurations to be tested to determine system interop- 
erability, becomes relatively large. 

[0017] The Session Initiation Protocol (SIP) Identifies the following functionality which Is needed to set up multimedia 
conferences: 

[0018] User Location: Determination of the end system to be used for communication. A location server provides 
the user location Information, in response to a request from a SIP server or proxy. Users may also register their locations 

with a SIP server, using a REGISTER method. 

[0019] User Availability: Determination of the willingness of a called party to engage In communications. An end- 
user wishing to set up a call with another end-user, or to add it to a conference. Invokes an INVITE method on the 
user's SIP server. Possible responses include "Success" and "Redirection". 

[0020] User Capabilities: Determination of the media and media parameters to be used. To determine what media 

and media parameters will be used for a call, the calling system Invokes an OPTION method on the SIP sen/er of the 

called party. This is an inquiry as to what are the capabilities of the called party. 

[0021] Call Setup: "Ringing", establishment of call parameters at both called and calling party. 

[0022] Inoperabillty of SIP implementations is less problematic than that of H.323 implementations. Interoperation 

of SI P user agents and servers with H.323 networks remains problematic, however. A type 7R/E Programmable Feature 

Server, available from Lucent Technologies Inc., provides protocol Intenworking through device servers that interface 

to specific protocols. 

[0023] The Media Gateway Control Protocol (MGCP) is a combination of two earlier standards, IP Device Control 
(IPDC) and SGCP (Simple Gateway Control Protocol SGCP). The protocol provides for media gateways that convert 
signals from TDM to packet, and back, and for call agents that control the behavior of media gateways. This is a master/ 
slave arrangement, with the call agent as the master and the gateway as the slave. Synchronization between various 
call agents involved In a call must be handled by other means. 

[0024] MGCP is thus used as an internal protocol within a distributed system that appears as a single VoIP gateway 
to an external H.323 network. A call agent may signal to other call agents using the RAS and H.225.0 protocols. This 
provides means to intenwork H.323 networks with IP networks. 

[0025] Accordingly, there is currently a need to verify that when two communication systems are connected for in- 
tenworklng, such as to provide, e.g., VoIP service, the connected system behavior will be as expected from interactions 
with one or more components of the integrated system under test. Specifically, there is a need for a method in the form 
of, e.g., portable software, that will generate test sequences which, if successfully implemented, will assure both branch 
and full path coverages for the connected systems. 

Summary Of The Invention 

[0026] According to the invention, a method of generating a set of test sequences to determine interoperability of 
communication systems when connected to one another, includes defining gateways between the connected systems 
and corresponding end users of the connected systems. Certain transitions are determined that involve operation of 
more than one gateway associated with a calling user and a called user, and the connected systems are tested for 
operation at those transitions. 

[0027] According to another aspect of the invention, a method of generating a set of test sequences to determine 
interoperability of communication systems when connected to one another, includes defining a state machine corre- 
sponding to the connected systems, the machine having a number of defined states, and transitions between the 
states. Transitions that Involve operation of more than one system gateway associated with a calling user and a called 
user are determined, and the connected systems are tested for operation at at least some of the transitions. 
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[0028] For a better und^anding of the invention, reference is made to the followingoescription taken in conjunction 
with the accompanying drawing and the appended claims. 

Brief Description Of The Drawing 

[0029] In the drawing: 

FIG. 1 is a blocl< diagram of a model showing voice end-users connected to one another by an IP Network through 
associated gateways; 

FIG 2. Is a typical finite state machine (FSM) representation of the interconnected model in FIG. 1; 

FIGS. 3-9 show various states, and transitions between the states in the FSM of FIG. 2; 

15 FIGS. 1 0-1 3 together represent a first step toward determining which transitions are to be tested for interoperability 

of interconnected communication systems; 

FIG. 14 represents a second step toward determining a set of tests for system interoperability; 

20 FIG. 15 represents a third step toward determining a set of tests for system interoperability; 

FIG. 16 is a block diagram representing criteria for a set of tests to ensure adequate coverage for interconnected 
communication systems; and 

25 FIG. 17 is a diagram representing a software program that produces test sequences for interconnected commu- 

nication systems. 

Detailed Description Of The Invention 

30 [0030] Interoperability testing is concerned only with failures that may occur when two different systems are con- 
nected or coupled with one another to interoperate. It is assumed that two systems operate correctly when running 
independently, or when coupled to other identical systems. 

Potential interoperation failures to be checked are those due to interoperation between two independently de- 
signed or developed systems. 

35 [0031] A model 10 for a general configuration of interoperating communication systems, such as Voice over Ip sys- 
tems, is shown in FIG. 1 . The figure shows two end-users 12,14 that want to communicate. Each user can access the 
systems through a corresponding gateway 1 6, 1 8. 

The two gateways 16. 18 communicate with each other using a defined protocol to decide whether and how to 
start the communication between the end-users 12,14. The protocol takes the gateways 16,18 through various states 

40 as they negotiate concerning the desired communication. The end users 12. 14 may wish to communicate by voice, 
or to exchange other information in either analog or digital 

[0032] It is important to note that interoperability errors, if any, will be introduced only when the gateways 16. 18 
actually communicate with one another about a call. Local activities involved in the protocol, e.g., obtaining information 
from either end-user 12 or 14 can be ignored. Interactions or "transitions" that can be ignored for interoperability pur- 
45 poses are referred to as "white", and all other transitions are referred to as "black". A white transition is purely local, 
that is, it reads the state of only one gateway 16 or 18, and writes the same state. A black transition involves both 
gateways 16 and 18 because the transition reads the states of both of the gateways, or reads the state of one and 
writes the state of the other. 

[0033] For example, an "off-hook" transition from an idle state of a user's telephone to a dialing state of the same 
50 user's telephone is white, because the transition involves only an originating gateway and its associated end user. A 
"dial" transition from "dialing" of a calling user's telephone to "ringing" of a called user's telephone is black, because 
the states change in both gateways 16 and 18. Similarly, an off-hook transition that connects the call changes states 
in both gateways 16,18 from ringing to connected. Accordingly, test cases that correspond only to sequences involving 
black edges are generated, i.e., those sequences that occur while a call Is under negotiation between the two gateways 
55 16 and 18. 

[0034] Expected system behavior is modeled in accordance with a finite state machine (FSM) 50, typical vertices 
(nodes) and edges of which are shown in FIG. 2. The FSM 50 of FIG. 2 has 21 states (nodes) and a total of 68 transitions 
between the states, as defined in FIGS. 3-9. A transition from a first state to a second state is identified by locating the 
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5t line of one of the 68 transitions in FIGS. 3-9. Ideally, affpossible execution sequences 
or "scenarios" should be covered. Because the transition diagram of the FSM 50 is a directed graph, covering all 
possible execution sequences requires that all branches and all possible paths be tested. Criteria for ruling out "re- 
dundant" scenarios are given further below, however. 
5 [0035] Generally, the transition diagram will contain cycles, and, therefore, will have an infinite number of distinct 
paths. Therefore, the test generation process includes the following three steps: 

Test Generation Process 

10 [0036] Step 1 : Generate all possible acyclic paths, i.e., paths without repeated vertices. See FIGS. 1 0 to 1 3. 

[0037] Step 2: Generate all possible simple cycles, i.e.. cycles that do not contain any smaller cycles. See FIG. 1 4; and 
[0038] Step 3: "Combine" the paths (from Step 1 ) and cycles (from Step 2) to generate a final set of paths. See FIG. 1 5. 
[0039] The number of paths and cycles generated in the first two steps is finite. Various criteria for removing redundant 
acyclic paths and redundant simple cycles are given later below. 

15 [0040] With respect to Steps 1 and 2. all strongly connected components (SCC) of the transition graph are first found. 
This has two advantages; (a) we know that any cycle is completely contained inside a SCC, so, Step 2 can be performed 
by looking at each SCC in turn and finding all simple cycles within the SCC, and (b) we can "shrink" each SCC into a 
node and obtain a Directed Acyclic Graph (DAG), i.e., a graph without any cycles. This translates into a two-phase 
process for Step 1. First, generate all acyclic paths on the resulting DAG, and then replace each SCC on any given 

20 path with a set of acyclic paths within the SCC. 

[0041] The goal of the "combine" process in Step 3 is to generate a finite number of paths that cover all scenarios 
of interest. This is done first by including all the acyclic paths. Then additional cyclic paths are generated as follows. 
[0042] For each acyclic path P, find all the cycles that share a node with P. Let these cycles be C^ .Cg, C^, and let 
Vi, 1 < i < k, be a node common to C| and P. Then generate a new (cyclic) path by replacing node Vj in P with cycle q. 
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Next-Transition-Tree 



[0043] A simple data structure, next-transition-tree, which is convenient for Steps 1 and 2, is now described. See 
FIG. 12. The data structure can be defined for any graph but, in the present application, the graph is always a SCC. 

30 [0044] For any node v, next-transition-tree(v) stores all acyclic paths from v to other vertices in its SCC. The tree has 
V as its root, and its height is equal to the number of nodes in the SCC containing v. Children of v are all the nodes In 
its SCC that have a direct edge from v. In general, the children of any node are all the nodes in its SCC that have a 
direct edge from node \i. Note that a node may appear multiple times in this tree. That is, each node has a label where 
the labels are not necessarily unique. The root node has label v. A node labeled has as many children as the outdegree 

35 of u (number of edges leaving u) in its SCC, and these children are labeled with the corresponding nodes in the SCC. 
[0045] For ease of illustration, assume that a separate next-transition-tree is maintained for each node in the graph. 
The actual implementation may have many shared pieces among next-transition-trees belonging to the same SCC. 



Generating All Simple Cycles 



[0046] All simple cycles containing any node v are generated as follows: Consider all paths in next-transition-tree(v) 
that contain another (than root) Instance of node v. In any such path, the path segment from the root to the first (closed 
from root) occurrence of v corresponds to a cycle containing v. This path segment will correspond to a "simple" cycle 
if it does not contain any repeated nodes. Therefore, the process includes finding all such path segments, and all 
45 simple cycles containing v are generated. As mentioned, this is a simplified description of an actual implementation, 
which should not maintain any paths with repeated nodes in the next-transition-tree. 

[0047] One way to generate all simple cycles in a SCC is to consider its vertices in some order: v^ , V2 V|^. Generate 

all simple cycles containing the node v^ . Then generate all simple cycles containing the node Vg that don't contain node 
v^. This can be accomplished by modifying the above procedure so that path segments containing the node v^ are 
50 ignored. Ail simple cycles containing the node Vg that don't contain nodes v^ or Vg. are then generated, and so on. 

Generating All Acrylic Paths 

[0048] As mentioned earlier, all the strongly connected components (SCC) of the transition graph must be found. A 
55 "component graph" where each node represents a SCC, and an edge from node u to node v represents an edge (in 
the original transition graph) from a node belonging to u's SCC to a node belonging to v's SCC, is then constructed. 
This component graph is a Directed Acyclic Graph (DAG), i.e., a graph without any cycles. See A.V. Aho, et aL, The 
Design and Analysis of Computer Algorithms (Addison-Wesley 1974), all relevant portions of which are incorporated 
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by reference. All acyclid^hs in the DAG are first generated, and then each SCC on any given path is replaced with 
a set of acyclic paths within the SCC, 

Redundancy 

5 

[0049] There are a finite number of distinct paths representing potential tests in a DAG. Not all of them need to be 
tested, however. Rather, a minimal number of tests, which collectively satisfy a coverage criterion for the connected 
systems and which do not contain any redundant tests, are desirably generated. 

[0050] The tests must start from a "Start" state, so only those paths beginning from a source node which has no 
10 incoming transitions are considered first. The following redundancy criteria may then be applied. 

(R1 ) Proper prefixes are redundant and may be ignored. 

[0051 ] Redundancy criterion R1 implies that only those paths terminating at a sink node (a node without an outgoing 
15 transition) be considered. Any path not terminating at a sink node is a proper prefix of a path that has been extended 
to a sink node. Accordingly, only those paths that start from the source node and terminate at one of the sink nodes 
need to be generated. 

[0052] The following procedure generates all source-to-sink paths in a DAG. 
20 PATHS-IN-DAG 

[0053] Input, a DAG G with one sources and multiple sinks. Output, all source-sink paths in G. 
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1 topologically sourt nodes in G: 5 = v^,v,,...,v^-l ; 

2 for i=n - 1, . . . , 0 

3 if V. is a sinJc node then 

4 p{^i)~{^}'^* ^ singleton set on empty path */ 

5 else 

6 let outgoing edges from be: 

7 p(vO = U',.,(v„>v^Xw,); 

8 return p(vo) 

45 [0054] The above procedure is bottom-up, starting from one of the sink nodes t = v^.^ . When processing a node Vj, 
we examine all of its outgoing edges (Vj,W|) where the paths from Wj to sink nodes have been computed. At Line 7 
above, we concatenate edge (Vj.Wj) to each path computed at W| and collect it at node Vj. After processing the source 
node s = Vq, all of the paths which are irredundant and have complete coverage are obtained. 
[0055] A topological sort takes time proportional to the number of edges. We charge to the examination and con- 

50 catenation of each edge, which is processed only once at Line 7, and the total cost is proportional to the total paths 
lengths. 

[0056] Proposition 1 . The procedure PATHS-IN-DAG constructs all source-sink paths. Its time and space requirement 
is linear in output, i.e., the total lengths of all the constructed tests. 

[0057] Proposition 2. For machines with a reset to source s, the constructed test sequences are a checking sequence. 
Generating Ail Acyclic Paths within a SCC 

[0058] The generation of paths (i.e., tests) for DAGs where nodes can be SCO's has been discussed. For each edge 
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connecting two SCC's o^path, the edge is replaced with an edge in the original grapFTln general, it will be possible 
to replace an edge between SCC's with one of several possible edges in the original graph, and a separate path for 
each choice of replacement edge can be obtained. 

[0059] Each SCC node is also replaced with a set of acyclic paths within the SCC. Assume the incoming and the 
5 outgoing edges to the SCC node are on nodes u and v of the SCC. The SCC node needs to be replaced with all 

possible acyclic paths from u to v in the SCC. These paths can be generated from next-transition-tree(u). 

[0060] Replacing any SCC with all possible acyclic path provides exhaustive coverage, but may generate a relatively 

large number of tests. In certain applications, it may not be necessary to cover the SCC's as thoroughly. Two options 

used in practice are as follows: 
10 [0061] Chinese Postman Tour. For an SCC, each edge (transition) Is tested at least once and the test sequence 

length is minimized. Such a path is called Chinese Postman Tour. See A. V. Aho, et al., An Optimization Technique for 

Protocol Conformance Test Generation Based on UIO Sequences and Rural Chinese Postman Tours, 39 IEEE Trans. 

on Communication, No. 11 (Nov. 1991) at 1 604-15, which is incorporated by reference. 

[0062] Checking Sequence. A more thorough coverage is provided by a checking sequence that guarantees the 
15 structural isomorphism of the implementation and the specification machines. The length of such a test sequence will 

be longer than that of a Chinese Postman Tour. 

[0063] The following procedure summarizes the steps in generating all possible tests: 



TEST-GENERATION 

20 

[0064] Input. An EFSM with a designated source state Output. Minimal set of tests providing full coverage 

1 Construct an equivalent FSM and its transition diagram G 

2 Find ail SCC's of G. 
25 3 For each SCC 

4 For each node in the SCC 

5 construct next-generation-tree 

6 Shrink each SCC og G to obtain a DAG G' 

7 Apply procedure PATH-IN-DAG on G' to obtain all acyclic source-sink paths in G' 
30 8 Replace each edge and node to obtain the set P of acyclic paths 

9 Obtain the set C of all simple cycles 

1 0 Combine the sets P and C to obtain the final set of tests 



Additional Redundancy Criteria 

35 

[0065] The TEST-GENERATION procedure, above, provides exhaustive coverage. The number of tests generated 
may be enormous even for moderate sized systems, however. 

[0066] As stated earlier, system interoperability errors may occur only when the system gateways actually "talk" to 
one another concerning a given call. Transitions are therefore labeled as either "white" to connote local activity, or as 
40 "black" if they involve both gateways. Thus, for interoperability test generation, white transitions need not be covered, 
and the following additional redundancy criteria become appropriate. 



(R2) Remove all-white test sequences. 

45 (R3) Let (u,v) be the last black edge in the test sequence. Then replace the path from the source node to node u 

with the shortest path between the source node and node u. 

(R4) Let (u,v) be the first black edge in the test sequence. Then replace the path from node v to the sink node with 
the shortest path between node v and the sink node. 

50 

(R5) If there is a sequence of white edges in which each party separately reaches an idle state, terminate the 
sequence at the last black edge and then use a shortest path to the state "IdleA, IdleB" at the end of the sequence. 

[0067] Criteria R2, R3 and R4, above, reflect that white transitions are not relevant for interoperability testing. Their 
55 only use is to connect relevant black transitions. For example, R3 states that the sequence of transitions before the 
first black transition is not relevant, so that part may be replaced with the shortest all-white sequence. Criterion R4 is 
a dual of R3. The last criterion R5 reflects that if both parties reach an "idle" state through a sequence of non-relevant 
transitions, then nothing concerning system interoperability will happen in the rest of the sequence. 



7 



EP 1 122 913 A2 

[0068] The following procedure generates all acyclic paths satisfying these criteriaf 

COMPLETE COVERAGE 

5 [0069] Input, transition graph G of a FSM. 

Output, complete set of paths according to redundancy criteria R1 -R5. 

1 for each node v that has an outgoing black edge 

2 if there is an all-white path from the source node to v then 
10 3 add a super-edge from the source node to v; 

4 delete all outgoing edges (not super-edges) from the source node; 

5 for each node v that has an incoming black edge 

6 if there Is an all-white path from v to a sink node then 

7 add a supper-edge from v to this sink node; 

15 8 delete all incoming edges (not super-edges) to the sink node; 

9 generate all acyclic paths in the resulting graph by procedure PATHS-IN-DAG with the modification that no super- 
edge should follow or precede a white edge; 

10 replace each super-edge with the shortest all-white path in the orginal graph* 

11 preocess each path according to redundancy criterion R5; 
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Generating Test Sequences according to R1-R5 



[0070] The above procedure COMPLETE-COVERAGE starts with a graph G and generates another graph G' such 
that the set of source-sink paths in G' is the same as the set of source-sink paths in G; while satisfying criteria R2-R4. 

25 Lines 1-4 of COMPLETE-COVERAGE consider all possible black transitions that may be the first in any sequence, 
and add a marker 'super-edge' for the shortest all-white path from the source node. Line 9 ensures that a black transition 
follows a super-edge. The marker 'super-edge' is replaced on line 10 by the shortest all-white path. This handles 
redundancy criterion R3. Lines 5-8 perform an analogous function for sink nodes and the last black transitions in the 
sequence, reflecting redundancy criterion R4. Criterion R2 is satisfied because line 9 ensures there is at least one 

30 black transition. Instead of generating all paths and then processing them according to criterion R5 per line .11, an 
actual implementation may ensure that the incremental construction of all paths in procedure PATHS-IN-DAG Is such 
that a path in violation of R5 is never generated. 



Proposition 3. The procedure COMPLETE-COVERAGE constructs all acyclic paths according to criteria R1-R5. 
Generating a Smaller Set of Test Sequences 



[0071] On most practical systems, we expect the above procedure COMPLETE-COVERAGE to generate a consid- 
erably smaller set of test sequences than the procedure TEST-GENERATION. But even such a smaller set may be 
40 too large for manual test execution, however Accordingly, the following describes a set of more restrictive criteria for 
test generation, concentrating only on black transitions. {R6) Generate acyclic paths having only black edges, except 
that the prefix from the source node, and the suffix to the sink node, are allowed to contain white edges. 
(R7) Generate simple cycles having only black edges. 

[0072] The following procedure ADEQUATE-COVERAGE generates a test-set according to R6 and R7. 
45 Input, transistion graph G of a FSM. 

Output, test generation according to R6 and R7. 
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1 full coverage - 

2 for each node v that has an outgoing black edge 

3 if there is a path from the source node to v then 

4 add a super-edge from the source node to v; 

5 for each node v that has an incoming black edge 

6 if there is a path from v to a sink node then 

7 add a super-edge from v to this sink node; 

8 delete all white edges from the graph; 

/* the only remaining edges are super-edge or black 
edges */ 

9. generate all acyclic paths in the resulting graph; 

10 replace each super-edge with the shortest path in the 
original graph; ■ 

[0073] The procedure ADEQUATE COVERAGE starts with a graph G, and transforms it into a graph G' such that 
the set of source-sink paths in G' is the same as the set of source-sink paths in G with criterion R6. The intuition of the 
procedure is to delete all white edges except those needed to reach black transitions from the source node, or from 
the black transitions to the sink nodes. Line [?] maintains a marker 'super-edge' for each source node to black transition 
path. On line 10, this marker is replaced with the shortest path. Lines 4 perform an analogous function for sink nodes. 
Criteria for adequate coverage are represented in FIG. 16. 

Proposition 4. The procedure ADEQUATE-COVERAGE constructs all acyclic paths according to criterion R6. 
[0074] If automated test execution is available, then the COM PETE-COVE RAGE procedure is more desirable. If 
tests must be executed by hand, however, then the ADEQUATE-COVERAGE procedure will likely produce a manage- 
able set of test sequences. Alternatively, one may always start with the ADEQUATE-COVERAGE procedure since it 
does apply to the most critical interoperability behavior. If the systems pass those tests generated by ADEQUATE- 
COVERAGE, one may obtain the broader coverage provided by the COMPLETE-COVERAGE procedure. 

EXAMPLE 

[0075] A portable software tool referred to as Interoperability Testing Intelligent System (IT-IS) for automated inter- 
operability test generation, was written in ANSI C and Tcl/Tk. The program includes a graphical user interface (GUI) 
for user input, and for displaying generated test sequences. The workflow of IT-IS is shown in FIG. 17. 
[0076] The input to IT-IS is an extended FSM (EFSM) description of the composed system behavior, IT-IS first per- 
forms teachableness analysis to convert the EFSM into a FSM, e.g., FSM 50 in FIG. 2, and then uses different proce- 
dures on the FSM to generate the test sequences. 

[0077] IT-IS was used to generate interoperability test cases for end-user VoIP testing. As shown in FIGS. 3-9, the 
FSM 50 has 21 states and 68 transitions. 

[0078] Among the transitions, 24 are colored black, others are white. The shrunk DAG generated by IT-IS from the 
FSM 50 contains 3 SCC nodes, and only one of them has more than one state. The number of test sequences generated 
by applying the procedures described above, is shown in the following table. 



55 


Procedure 


Loop-Free Paths 


Loops 


Final Tests 




TEST-GENERATION 


950 


424 


1752 




COMPLETE-COVERAGE 


508 


424 


908 
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(continued) 




Procedure 



Loop-Free Paths Loops Final Tests 



ADEQUATE-COVERAGE 



16 4 22 



The final tests for either of the complete-coverage or the adequate-coverage procedures Involve only black transitions. 
[0079] Interoperability testing is indispensable for integration of reactive communication systems. The presently dis- 
closed procedures are distinguishable over conventional conformance testing techniques in that the procedures focus 
on the system interfaces, and are not directed solely to individual system implementations or specifications. 
[0080] While the foregoing description represents preferred embodiments of the invention, it will be obvious to those 
skilled in the art that various changes and modifications may be made, without departing from the spirit and scope of 
the invention pointed out by the following claims. 



1. A method of generating a set of test sequences to determine interoperability of communication systems when 
connected to one another, comprising: 

defining gateways between the connected systems and corresponding end users of the connected systems; 

detennining certain transitions that involve operation of more than one gateway associated with a calling user 
and a called user; and 

testing the connected systems for interoperation at said certain transitions. 

2. A method of generating a set of test sequences to determine interoperability of communication systems when 
connected to one another, comprising: 

defining a state machine corresponding to the connected systems, the machine having a number of defined 
states, and transitions between the states; 

determining which of the transitions involve operations of more than one system gateway associated with a 
calling user and a called user; and 

testing the connected systems for operation at at least some of said certain transitions. 



Claims 
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FIG. 3 



gui efsm 

states 1 21 
1. CallingAColledB 
2. 



3. IdleABusyB 

4. CalledACallingB 

5. Bus/ABusyB 

6. DiolingAUmboB 

7. BusyADiolingB 

8. BusyAIdleB 

9. BusyAUmboB 

10. LimboADialingB 

11. ConnectedCdledAConnectedCallerB 

12. UmboABusyB 

13. IdleADiolingB 
U. UmboAIdleB 

15. DialingABusyB 

16. HoldColledAHoldCailerB 

17. HoldCollerAHoldCalledB 

18. DiaiingAIdleB 

19. IdleALimboB 

20. ConnectedCollerAConnecUdCailedB 

21. IdleAIdleB 



i = input 

0 = output 

p = predicate 

a = action 



start I 
IdleAIdleB 

boolean term=folse; 
boolean IdleA=true; 
boolean IdleB=tnie; 
boolean DialingA=false; 
booleon DlalingB=folse; 

I 

transitions | 68 

1. BusyADiolingB BusyAIdleB \ 

\ I jOn-hook Bi | 

oiin 

p I true ( 

a j I D'ralingBsfolse; IdleB=true | j 

I 

2. IdleADiolingB DiolingADiolingB | 

i I |Of{-hool< Aj I 

0 I iDialTone Ai | 
p I true I 

a I |IdleA=false; Dlalin9A=tnje| | 

\ 

3. BusyADiolingB IdleADiolingB | 

1 I jOn-hook A| | 

oiin 

p { true I 

0 I } IdleA=:tnje | } 

\ 

4. IdleABusyB IdleAIdleB j . 

1 I jOn-hook B} | 

ollil 
p I true I 

a { jtcrm=true;IdleB=true{ | 
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FIQ. 4 



\ 

5. HoldCollerAHoldColledB IdleAIdleB { 11. 

i i jOn-hook A| } 

olHI 
p } tn/8 1 

a j |term=tnje;IdleA=true;IdieB=true{ | 

( 

6. IdieAUmboB IdleADIalingB i 12. 

i I jlimeout Bj \ 
0 j jDiolTone B| | 
p I true I 

a I I OialingB=tnie | | 



7. BusyADialingB BusyABusyB | 
i I {Dial B A| I 
0 1 jUneBusyTone B| | 
p I true I 

a j I DialingB=false j | 



13. 



14. 



8. HoldCalledAHoldCallerfi IdleAIdleB { 

1 I jOn-hook B| j 

oHIl 

p I tnie I 

Q I |term=true;IdleB=true;IdleA=true| { 

I 

9. OialingAIdleB DiallngADialingB | 15. 

i j |Off-hook Bi I 

0 j jDialTone B| | 
p I true I 

a i I IdleB=false; DialingB=true | | 

{ 

10. IdleADIalingB IdleAIdleB i 16. 

1 I {On-hook B| \ 

oilll 
p I true I 

0 I |term=tnje;IdleB=true;Dial]ng6=false} | 



} 

BusyABusyB BusyAIdleB | 
i I jOn-hook B| | 

oil! I 
p I true I 

d I i IdleB=true | | 

I 

DiallngADialingB BusyADialingB | 
i 1 iDial A Bi | 
0 1 jUneBusyTone A| j 
p { true { 
a I DialingA=false | 

} 

IdleAIdleB IdleADIalingB | 
i } jOff-hook B| I 

0 j iDialTone Bj j 
p j Iterm } 

a i |IdleB=false; DialingB=truej | 

\ 

HoldCalledAHoldCallerB IdleADIalingB | 

1 I jlimeout AB| | 

0 i |DialToneB| j 
p { true I 

a I I IdleA=true; DialingB=tnje \ \ 

I 

IdleABusyB DiolingABusyB | 

1 I joff-hook A| I 

0 j iDialTone A| | 
p I true I 

a { I IdleA=false; DialingA=tnje | | 

I 

CallingACalledB ConnectedCallerAConnectedCalledB | 

1 j jOff-hook Bj \ 

ollil 
p i true I 

oilll 
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FIG, 5 



I 

17. DialingADiolingB DialingABusyB | 

i I iDial B A| I 
0 I jUneBusyTone B| j 
p I true i 

a I { DialingB=false | | 

! 

18. CalledACallingB IdleAUmboB j 

; i {Timeout AB} | 

ofin 

p I true I 

0 I i IdleA=true j } 

I 

19. ConnectedCallerAConnectedCalledB IdleAUmboB } 

1 { lOn-hook Ai | 

oiin 

p I true { 

a } I ldl8A=true { | 

\ 

20. ConnectedCallerAConnectedCalledB HoldCallerAHoldCalledB { 

i 1 jOn-hook B| | 
oHIl 

p j true 1 I 
aiili 



21. IdleAIdleB DialingAIdleB \ 

i I jOff-hook A{ I 
0 I joialTone A| | 
p I Iterm j 

0 I |IdleA=false; DiQlingA=true{ j 

1 

22. DialingABusyB BusyABusyB | 

1 I iDial A B| ! 

0 { {LineBusyTone Aj | 
p I true I 

a I I OiarmgA=false | | 



23. IdleAUmboB IdleAIdleB | 
i } jOn-hook B| | 

oil! I 
p { true { 

Q I |t€rm=true*JdleB=true( | 

I 

24. UmboAIdleB IdleAIdleB | 

i I }On-hook Aj | 

o{in 

p I true I 

a j jterm=true^dleA=truej \ 

\ 

25. HoldCalledAHoldCallerfi ConnectedCalledAConnectedCallerB | 

i } jOfi-hook Aj i 

oiin 

p { true I 

oHIl 



19 



EP 1 122 913 A2 



! 

26. DialingABusyB IdleABusyB | 

i I |On-hook A| | 

olid 

p I true I 

a i I DiQlingA=false; IdleA=true \ \ 

I 

27. DialingABusyB DialingAIdleB | 

i j {On-hook B| | 

o\\\\ 

p I true { 

a j I IdleB=true ( | 

I 

28. HoldCoilerAHoldCalledB DialingAIdleB | 

i I {Timeout AB} ( 

0 1 iDialTone A| | 
p I true { 

a } { DialingA=true; IdleB=true | ( 

1 

29. DialingAIdleB CalllngACalledB | 

1 I {Diol A 6( i 

0 I {AudibleRinging A,Ringing B| | 
p I true I 

a i |Dial!ngA=false: IdleB=false| { 

I 

30. IdleADialingB CalledACallingB | 

i } jDiai B Aj | 

0 I jAudlbleRinging B,Ringing Aj | 
p I true { 

a I I IdleA=false; OialingB=false | | 

} 

31. BusyABusyB IdleABusyB \ 

1 \ iOn-hook Aj } 

ollil 
p I true j 

a j I IdieA=true | | 



FIG. 6 



\ 

32. CalledACollingB IdleAIdleB | 

i I jOn-hook B| | 

oiin 

p I true I 

Q j jterm=true;IdleA=true;IdleB=true| | 

I 

33. ConnectedCalledAConnectedCallerS HoldCalledAHoldCoilerB | 

1 1 {On-hook A| | 

clill 
p I true I 

oil I 

I 

34. BusyAIdleB BusyADialingB { 

i { {Off-hook B| I 

0 { {DialTone B| | 
p { true i 

a I { IdleB=false: DkilingB=true ( | 

! 

35. UmboAIdleB DialingAIdleB { 

1 { {Timeout Aj \ 

0 { {{KalToneAj | 
p I true { 

Q { { DialingA = true; { j 

i 

36. IdleAUmboB DialingALimboB { 

1 { {Off-hook Aj j 

0 { {DialTone Aj j 
p { true j 

a { {IdleA=false;DialingA=(njej j 

I 

37. UmboAIdleB UmboADialingB { 

1 { {Off-hook Bj j 
0 { {DialTone Bj j 
p { true I 

a { {IdleB=false;Dialing6=true| | 
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FIG, 7 



} 

38. DiolingALimboB IdleAUmboB | 

i 1 }On-hook A} | 

olid 
p { true I 

a I {IdleA=true;DiaimgA=fal5e| | 

I 

39. BusyALimboB IdleAUmboB | 

i { {On-hook A| | 

p { true j 
Q I IdleA=true | 

I 

40. LimboABusyB IdleABusyB | 

i I }On-hook Aj } 

p j true j 
a I IdleA=true \ 



I 

44. BusyALimboB BusyAIdleB | 

i I jOn-hook B} \ 

oHIl 
p i true I 
a j IdleB=true ( 

( 

45. BusyAIdleB IdleAIdleB I 

i i {On-hook Aj j 

oiin 

p I true I 

Q j {term=true;IdleA=true| j 

I 

46. UmboADialingB UmboAIdleB { 

i I |On-hook B| { 

oiin 

p I true I 

0 I iDialingB=false;IdleB=tnie{ | 



47. 



48. 



41. UmboAOiolingB IdleADialingB { 

i i jOn-hook Aj \ 

oHIl 
p j true i 
Q { IdieA=true | 

I 

42. DiolingALimboB DiolingAIdleB | 

i I jOn-hook B| j 

oil!! 
p I true { 
a } IdleB=tnie \ 

\ 

43. DiolingAIdleB IdleAIdleB } 

i I jOn-hook A| | 

olIM 
p I true I 

a. I |term=trueJdleA=true;DialingA=false| | 



49. 



LimboABusyB LimboAIdleB j 
i I iOn-hook Bj | 

oiin 

p { true \ 
a I IdleB=true | 

i 

ColledACollingB ConnectedColledAConnectedCollerB { 
i i lOff-hook A( I 

oiin 

p I true j 

0 nil 

i 

DiolingALimboB BusyALimboB | 

1 1 {Dial A Bj | 

0 \ {LineBusyTone A| | 

p { true } 

Q { Dial(ngA=false | 
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FIG. 8 



I 



50. 



51. 



i j jOn-hook A} { 

p j true } 
a { {IdleA=tnje; 



I 



i I iDial B A| ! 

0 \ jLineBusyTone 8} { 

p I true I 

0 j Dialing6=false { 



52. CollingACalledB UmboAIdleB { 
i i {Timeout ABj { 

p ] tnie i 

a i I IdleB=tnje; | | 



53. 



\ 



I 



i I jlimeout A| | 

0 \ joialTone A| | 

p j true I 

a \ DiQlingA=true \ 



54. DialingADialingB DiallngAIdleB \ 

i j jOn-hook B| i 

oiin 

p I true I 

Q I j DiQlingB=fQlse; IdleB=true j ( 

1 

55. BusyADialingB DialingADialingB { 

i i {Timeout Ai ( 

0 I iDiolTone A( | 

p j true j 

a I DialingA=true } 



56. ConnectedCalledAConnectedCallerB LimboAIdleB { 

i j iOn-hook B| | 

ollll 
p ! true { 

a i i IdleB=true \ \ 

I 

57. Bus/AUmboB DiolingAUmboB { 

i j {Timeout Aj i 
0 j {Diallone Aj } 
p j true I 

0 { DiailngA=true | 

1 

58. HoldCflllerAHoldCalledB ConnecledCallerAConnectedCalledB | 

1 I jOff-hook B} j 

oiii! 
p { true j 

oiin 

I 

59. BusyABusyB DialingABusyB { 

i I {Timeout A( | 
0 j joialTone A} \ 
p I true I 
a I DialingA=true j 

{ 



60. 

i I jOnhook A| ( 

oilM 
p { true I 

a j {term=tnje;IdleA=true*JdleB=truei | 

.! 

61. limboADialingB DialingADialingB { 
I { {Fimeout A| ( 
0 j {DialTone Aj j 
p { true i 
G I DialingA^true | 
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FIG. 9 



62. LimboABusyB DialingABusyB | 



68. UmboABusyB LimboADialingB \ 



i I jTimeout A| j 
0 I jDialTone A| | 
p I true I 



i I jTimeout B| } 
0 I jDiolTone B| | 
p I true I 




Q I DialingB=true j 



63. IdleABusyB IdleADialingB | 



i I jTimeout B| | 
0 I jDialTone B| | 
p j true I 



a j DidingB^true | 



64. DialingABusyB DialingADialingB | 
i j jTimeout B| i 
0 j jDialTone B| | 
p j true I 
0 j DialingB=true { 



65. DialingAUmboB DialingADialingB j 
i { jTimeout Bj i 
0 j {DialTone Bj | 
p I true I 
a j Dialin9B=true \ 



66. BusyABusyB BusyADIaltngB | 
i I jTimeout Bj | 
0 j jDialTone Bj } 
p j true I 
0 I DiaiingB=true \ 



67. BusyAUmboB BusyADiolingB | 
i j jTimeout Bj \ 
0 j jDialTone B| | 
p j true I 
a j DiolingB^true { 
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FIG. 17 
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